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MEMORANDUM FOR UNDER SECRETARY OF DEFENSE (ACQUISITION & 

TECHNOLOGY) 

SUBJECT: Report of the Defense Science Board Task Force on 
Acquiring Defense Software Commercially 

I am pleased to forward the report of the Defense Science 
Board Task Force on Acquiring Defense Software Commercially. The 
Task Force, co-chaired by Dr. George H. Heilmeier and Dr. Larry 
Druffel, was chartered to determine the conditions under which 
defense software could be procured using commercial practices and 
to develop a strategy for procurement that incorporates such 
practices. 

In its investigation into applying commercial practice to 
DoD software procurement, the Task Force reviewed a broad 
spectrum of interrelated elements from software program 
management policy and DoD software acquisition practices to DoD's 
investment in the software technology base. I wholeheartedly 
concur with the group's key finding that DoD must develop a more 
coordinated approach to the oversight of its diverse software 
capabilities and programs. The Task Force's report provides an 
outstanding point from which to begin addressing revamping DoD's 
software procurement practices. 



Paul G. Kaminski 
Chairman 
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DEFENSE SCIENCE 
BOARD 

Dr. Paul Kaminski 

Chairman, Defense Science Board 

Office of the Undersecretary for Acquisition and Technology 
The Pentagon 
Washington, D.C. 

Dear Dr. Kaminski: 

Enclosed is the final report of the Defense Science Board Task Force on Acquiring Defense 
Software Commercially. We were tasked to determine the conditions under which procurement of 
defense software can use commercial practices and to define needed changes to permit such use. 
In the attached report, we make specific reconmiendations with regard to DoD process credibility, 
software program management, required expertise of DOD personnel using modern software 
practices, use and integration of commercial off-the-shelf software, DoD software acquisition 
practices, use of software architectures by DoD as a management tool, and DoD's investment in the 
software technology base. 

Based on our review, we concluded that issues associated with defense software are 
applicable across the wide spectrum of DoD software intensive systems. However, to reap 
maximum benefit from improvements related to the myriad aspects associated with software, the 
DoD requires a more coordinated approach to the oversight of its diverse software capabilities and 
programs. 

To provide such Department-wide oversight and to facilitate implementation of the other 
specific recommendations contained within this report, the Task Force recommends that the 
Secretary of Defense assign to the Under Secretary of Defense (Acquisition and Technology) the 
responsibility for DoD-wide software technology, policy, practices and acquisition. This 
centralized approach will best serve the Department for all software intensive systems and 
programs. To support the implementation of this recommendation and to ensure that the key 
stakeholders are participants in the process, the Task Force further reconmiends the formation of 
an Executive Council consisting of the appropriate principals from OSD, the Services, and the 
Defense Agencies. 

We thank all of the members and government advisors of this Task Force for their 
dedicated efforts and significant contributions to this study. 
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Executive Summary 



The Defense Science Board Task Force on Acquiring Defense Software Commercially recognizes 
that DoD systems are becoming increasingly dependent on the use of software as the mechanism for 
implementing operational capabilities. To adapt to changing military and national security situations, DoD 
is more dependent than ever on its ability to modify mission software rapidly, often in near real-time. 
However, software remains the schedule and cost driver for the development and maintenance of many 
important defense systems. 

In its review of current DoD and conmiercial software acquisition practices, the Task Force found 
notable differences, as evidenced in Appendix C. There are, however, many similarities between the 
various categories of DoD and commercial software systems. Although there are indications that 
commercial development efforts have achieved better predictability and lower costs, the Task Force noted a 
significant lack of credible, quantitative data to substantiate this assessment. 

In general, the Task Force concluded that DoD's investment in software requires greater DoD-wide 
management control and oversight in the coming years if the Department is to exploit the use of 
commercial software acquisition practices fully, as well as rapid advances in software technology. The 
following is a summary of selected findings and recommendations toward that end. 

Process Credibility: Current DoD practice is not compatible with commercial business 
practices. DoD should work to make necessary changes to acquisition regulations such as: 

• Having program managers manage 3 of 3 (price/schedule/fiinctionality) but only constrain 2 of 3 

• Defining successful performance on contracts as delivering a solution (with predictable price, 
schedule and functionality) not adherence to government processes, procedures and specifications 

• Not requiring c-level specifications for software projects developed in Ada 

• Establishing mechanisms to allow both current ability to perform as well as past performance as 
key factors in source selection 

• Encouraging offerors to demonstrate as much functionality as possible as part of bid without 
eliminating domain knowledgeable competition 

DoD Program Management: DoD program management approaches discourage the use of 
commercial practices. Program managers lack incentives to tailor procedures to fit individual program 
needs or to develop "corporate" solutions (e.g., employ common architecture or common software 
components). DoD should establish and implement overarching software life cycle guidelines more 
conducive to the use of commercial practices and products, such as: 

• Defining software architectures to enable rapid changes and reuse 

• Facilitating early system engineering and iterative development 

• Participating in development of commercial and international standards 

• Allowing the fielding of software directly from test beds with user consent 

• Requiring program managers to stay with programs at least through beta testing to maintain 
continuity of understanding of original nuances in requirements 



DOD Personnel: There is currently a shortage of sufficiently qualified software personnel at all 
levels within the Department. DoD should establish a Department-wide software program management 
education and training initiative that includes: changing courses for PMs to reflect best commercial 
practices and other reconmiendations of this task force and providing for changes to reflect the dynamics 
of the software industry; rotating government and contractor personnel between PM and developer 
organizations to build understanding and trust; encouraging use of IPA's from industry; and integrating 
software-qualified personnel into senior DoD acquisition staff. 

Use and Integration of Commercial Off-the-Shelf (COTS) Software: DoD has not 

fully identified the pros and cons associated with the use of COTS software and, as a result, has not 
determined when and how best to use COTS software. To facilitate this process, DoD should require 
trade studies and analysis of the use of COTS software in DoD's software acquisition process where 
appropriate. Further, DoD should establish "customer friendly," application-specific information 
technology "component stores" to enable program managers to. assemble systems rather than develop them 
through use of reusable, prequalified components. DoD should also increase technology base funding for 
security audit tools for systems employing COTS software and should capitalize on innovative cost- 
effective techniques for acquiring and using COTS software products, such as the use of enterprise 
licenses. 

Software Architecture: Software architecture was emphasized by the Task Force as a means 
for achieving important ends. There is currently little emphasis on architecture in DoD software programs 
or regulations. As a result, DoD is not benefiting from architecture as a key tool for evolutionary 
development and for early (and frequent) involvement of users with functional capability and facilitating 
reuse, requirements changes with minimum cost and schedule, and "product line" management. DoD 
should require vendors to propose, manage and control the architecture and should establish an early 
architecture deliverable in all developments. 

Software Technology Base: The current DoD software technology base investment does not 
adequately take advantage of conmiercial R&D. Further, software technology transfer (both internal and 
external) is not receiving adequate emphasis within DoD. DoD should provide for the evolution of the 
DoD Software Technology Strategy to align with emerging conunercial technology and practices. 

Overarching Recommendations: To facilitate implementation of the many recommendations 
contained in this report, the Task Force concluded that DoD's investments in software require greater 
management control and DoD-wide oversight. To this end, the Task Force reconmiends that the Secretary 
of Defense (SECDEF) assign the Under Secretary of Defense (Acquisition and Technology) (USD (A&T)) 
the responsibility for DoD-wide software technology, policy, practices and acquisition. In carrying out 
this responsibility, the Task Force recommends that the USD (A&T) consider forming an executive 
council with the Director, Defense Research and Engineering (DDR&E), the Assistant Secretary of 
Defense (Command, Control, Communications and Intelligence) (ASD (C3I)) and appropriate 
representatives from the Services and Defense Agencies as members. Further, USD (A&T) should 
provide for a supporting "process action team" to assist in implementation of the recommendations of this 
Task Force study. 
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Defense Science Board Task Force 
on Acquiring DefetisiB Software Commercially 



1.0 TASK FORCE OVERVIEW 



1.1 TERMS OF REFERENCE 




Terms of Reference 



Objectives: 

• Determine: 

- Conditions Under which Procurement of Defense Software Can Use Conunercial Practices 

- Changes Required to permit Such Use 

• Develop Strategy that Incorporates Such Practices 

- Not Constrained by Existing DoD Standards 

- Viewed as Coexisting Alternative Strategy 

- Includes DoD Use of Commercial Software Products 

• Compare Proposed Strategy with Current DoD Strategy, Indicating 
Circumstances Where Each is Most Beneficial 



Appendix A provides the Terms of Reference by which the Task Force was estabUshed. The 
objectives and scope of this Task Force are outlined above. At the first meeting of the Task Force, the 
sponsor of the study (the Director, Defense Research and Engineering) requested that the Task Force 
provide a strategy that: was sensible, pragmatic, and unconstrained by current DoD acquisition practices; 
was based on evaluation of mechanisms for integrating defense software efforts with commercial software 
efforts; did not require legislative relief; and addressed the full spectrum of DoD software appUcations. 



Scope; 

• All Software Intensive Systems 

• All Stages of the Software Life-Cycle 
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1.2 CAVEATS 



Caveats 



• Relied on Inputs from DoD and Industry Experts 

• Provided No Detailed, Quantitative Assessment or 
Evaluation of Individual Topics 

• Did Not Address: 

- Success or Failure of Ada 

- Recommended Software Development Approach for Specific 
Programs 

- Specific COTS Products for DoD to Exploit 

- Trade-Off s Between Hardware and Software 



The Task Force relied heavily on inputs from a variety of DoD and industry experts regarding a 
wide range of topics related to defense software technology, policy, practices and acquisition. Appendix 
B lists the many briefings that were provided. Although the Task Force chose not to provide detailed, 
quantitative assessments or evaluation of individual topics, it used the information associated with these 
topics in the formulation of findings and recommendations. The areas not addressed by the Task Force, 
while important in and of themselves, were determined not to be directly germane to the development of an 
overall defense software strategy. 
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1.3 TOPICS ADDRESSED 



Topics Addressed 



Topic 



# DoD Software Acquisition Policies 



• Affordabflity 



PoD Management of Commercial Development Process 



Software Risk Management Techniques and Supporting Tools 



Minimum Effective Delivery Time 



Maintenance After Product Delivery 



Post-Deployment Product Enhancement 



Software Process Support Tools 



Quality 



Assured Availability 



Use of Development and Maintenance Tools 



Contracting : 

Technical Data Rights 



• Intellectual Property Rights 



Liability" 



» Alternative Forms of Procurement Agreements 



m Incentives for CreatioqAJse of Reusable Software Components 



Economic Incentives 



Technical: 
0 Importance of Software Architecture 



• State-of-the-Art and Best Commercial Practices 



# Development Tools 
9 Reusable Software Components 



• Technique^ools for Tailoring Commercial Components for 

Defense Use 



In its efforts to assess the appropriateness of DoD use of commercial practices, the Task Force 
addressed software management, contracting, and technical issues. The above viewgraph lists the primary 
topics considered within each of these categories. 
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1.4 MEMBERSHIP 



Members 



Co-Chairmen: 

• Dr. George H. Heilmeier 

• Dr. Larry Druffel 

Members: 

• Dr. Jacques Gansler 

• Mr. Jack Hancock 

• Mr. Patrick Hillier 

• Mr. Arthur E. Johnson 

• Dr. Bruce Johnson 

• Mr. Alan McLaughlin 

• Dr. Alvin E. Nashman 

• Mr. John Stenbit 

• Dr. Terry Straeter 

Independent DSB Reviewers 

• Mrs. Joan Habennann 

• Mr. Philip A. Odeen 

Executive Secretary 
Ms. Virginia L. Castor 

DSB Secretariat Representative 

• CDR Robert C. Hardee 



Bellcore 

Software Engineering Institute 
TASC 

Retired Exec VP, Pacific Bell 
EDS 

Loral Federal Systems 
Andersen Consulting 
MIT Lincoln Laboratory 
Computer Sciences Corporation 
TRW 

GDE Systems, Inc. 

Logistics Management Institute 
BDM International, Inc. 

ODDR&E/AT 
DSB 



Government Advisors 



OSD: 

• Ms. Deborah Castleman 

• Mr. Frank Kendall 

• Mr. Gene Porter 

• Mr. William Mounts 

• Ms. Linda Brown 

• Mr. Chris Dipetto 



OASD(C3I) 

OUSD(A&T) 

OUSD(A&T) 

ODUSD(AR) 

ODASD(IM) 

OUSD(A&T) 



Toint Staff; 

• Mr. Joseph Toma 



J6A 



Services; 

• LTG Peter Kind Army 

• RADM John G. Hekman Navy 

• Mr. Lloyd K. Moseinann II Air Force 



Agencies: 

V« Ms. Belkis Leong-Hong 
• Dr. Ed Thompson 

Task Force members represented a valued cross section of software expertise within both the DoD 
and commercial sectors. The government advisors represented senior executives (including the three 
Service Software Executive Officials) from the major software management organizations within the 
Department. 



DISA 
ARPA 
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2.0 CURRENT POD AND COMMERCIAL SOFTWARE ACQUISITION PRACTICES 

2.1 BACKGROUND 



Background: Previous Studies 



DSB Summer Study on Technology Base (1981) 

Joint Service Task Force on Software Problems (1982) 

AF SAB High Cost and Risk of Mission Critical Software (1983) 

CODSIA Report on DoD Management of Mission-Critical Computer Resources (1984) 
DSB Task Force on Military Software (1987) 
Ada Board Response to DSB Task Force (1988) 

Summer Report on Defense-Wide Audit of Support for Tactical Software (1988) 
Workshop on Executive Software Issues (1988) 
Workshop on Military Software (1988) 
Army Materiel Command Study (1989) 

Software Technology Development and Deplo3rment Plan for DoD Technology Base 
(1989) 

AF SAB Adapting Software Development Policies to Modem Technology (1989) 

Draft DoD Software Master Plan (1990) 

Draft DoD Software Technology Strategy (1991) 

AF SAB Study on Information Architecture (1993) 

Study on Military Standards Impacts on the Acquisition Process (1993) 

Draft Software Action Plan Working Group Report (1993) 

Evolutionary Acquisition Study, AFCEA (June 1993) 



As a point of departure, the Task Force noted a number of previous studies addressing issues 
related to the defense software technology, policy, practices and acquisition. Despite the increased 
emphasis given to software issues by the DoD (as evidenced by the above list), the majority of the 
recommendations resulting from these studies have not been implemented. 
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Background: Impediments to Change 



DoD Software Management 

No Single Operative Mechanism Exists to Implement Change 

Three Separate Domains: MIS, C3I, Embedded 

Two Separate Acquisition Management Structures 
Bureaucratic "Turf'' Inhibits Real Reform 
Acquisition Process 

5000 and 8000-Series Developments Typically Employ "Waterfall" 
Approach; Not Incremental/Spiral Approach 

Culture 

Systems Are Stove Pipe — My System, My Program (PM is King) 
DoD Acquisition Training Reinforces the Wrong Approaches 

Procurement 

The Contracting Pi^ocess Inhibits Creativity and Investment by 

Contractors; Limits Options 

Interpretation of Competition in Contracting Act 



To ensure that its recommendations could be readily implemented by the DoD, the Task Force 
identified the primary reasons why recommendations from previous studies had not been acted upon. The 
Task Force then formulated its recommendations to appropriately address these impediments. 
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Why Is This Study Important? 



m DoD Military Capability Increasingly Software Dependent 

• Software One of Few DoD Budget Areas That is Projected to Grow 

- Software Has Become the Overall Defense System Schedule and Cost Driver 

» Declining DoD Budget - Affordability a Major Concern 
» Escalating Costs of Software Development and Life Cycle 

- Significant Level of DoD Resources To Be Spent on Inf oimation Technology: 

» FY92-FY94: -$10 Billion/Year* 

» EIA Forecast for FY95 - FY98: -$10 Billion/Year 

» In-House vs. Contracted Out: 30% In; 70% Out 

- Will Require Greater Management Control 

• Rich Commercial Base to Tap; Many Opportunities 

- Custom Software - Acquisition Methodologies 

- Off-the-shelf Products - Approach to Requirements Determination 

• Functionality and Flexibility More Embedded in Software than Hardware 

• study is Timely Because DoD Leadership is Focused on Acquisition 
Reform 

- Something May Actually Get Done *Source: EIA 



Given its increasing reliance upon software as the mechanism for implenienting system 
capabilities, coupled with the rising costs associated with software development and maintenance, DoD 
must take action now to address the issues associated with software acquisition. The commercial sector 
provides numerous opportunities upon which the DoD could readily capitalize. The time is ripe for 
assertive DoD action, particularly since the current leadership is so strongly focused on acquisition reform. 
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The Software Domain 
is Very Large 



CUSTOM 
COMMERCIAL 
REAL TIME SYSTEMS 



LARGE DOD EMBEDDED REAL 
TIME SYSTEMS 



COMPLEXITY 




71^ 



COMMERCL\L 
ENGINEERING 
SYSTEMS 



SMALL DOD 
EMBEDDED REAL 
TIME SYSTEMS 



DOD 
ENGINEERING 
SYSTEMS 



CUSTOM DOD 
BUSINESS SYSTEMS 



CUSTOM COMMERCL\L 
BUSINESS SYSTEMS 



COMPLEX 
COMMERCIAL 
PRODUCTS 



SMALL 
COMMERQAL 
COTS PRODUCTS 
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COST TO DEVELOP 



The software domain reviewed by the Task Force encompasses a wide variety of DoD systems, 
ranging from software tools to large embedded real-time systems. The associated cost, complexity, and 
time required to develop these systems vary widely, both for conmiercial and military applications. 
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Background: 
DoD Software Acquisition Management 

Two Different Sets of Software Policies/Rules Managed by 
Two Different Organizations 

- USD (A&T) for Software Embedded in Weapons/Systems 

- ASD (C3I) for C3I Software and MIS Applications 

Majority of Software Issues are Applicable to All DoD 
Systems 

Growing Similarity Between DoD and Commercial Software 
Developments Across the Various Types of DoD Software 

- Modem Software Tending to Blur Distinction Between DoD and 
Commercial Applications 

Central DoD Leadership Needed More than Ever 

- Major Revisions Needed in DoD Software Policies/Rules and 
Management Across All DoD Applications in Order to Meet SECDEF's 
Acquisition Reform Goals 

- Interpretation and Implementation of DoD-Wide Policies/Rules by 
Management is a Central Issue 



Today, the management of DoD software acquisitions is quite complex. There are two sets of 
software policies/rules managed by two different organizations 

• USD (A&T) for software embedded in weapons/systems 

• ASD (C3I) for C3I software and MIS applications 

There currently exists within the DoD a dichotomous organizational structure for the management 
of DoD software intensive systems. The Under Secretary of Defense (Acquisition and Technology) is 
responsible for weapons systems; the Assistant Secretary of Defense (Command, Control, 
Communications, and Intelligence (C3I)) is responsible for C3I systems and Management Information 
Systems (MIS). However, as technology has evolved over the years, the issues associated with the use of 
software in all of these systems have become essentially the same. The issuance of different software- 
related policies and regulations that are oriented according to the type of system is no longer appropriate. 
If the DoD is to adequately address this fundamental issue, central focused management is required. 
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2.2 COMPARISON OF DEFENSE AND COMMERCIAL SOFTWARE 



Comparison of 
Defense and Commercial Software Needs 

• Defense and Commercial Software Applications Needs Are Merging 





Breadth of 
Application 


Typical Size 
(SLOq 


Upgrade/ 
Change 


Sensitivity to 
Errors 


- DoD Real Time Software 


Unique 


400,000 


Complex; 
Inflexible 


High 


- commicial Custom bottware Integrated 
Into Real-Time Operations 


Umque 


400,000 


Complex; 
Inflexible 


High 


Command and Control Applicarions 
- C3I 


Moving Toward 
Wide 


200,000 


Complex; Flexible 


Moderate 


- Commercial Custom bottware Integraied 
Into Business Operations 


Movmg 1 oward 
Wide 


200,000 


complex; Moving 
Toward Flexible 


Moderate 


MIS Systems 

- DoD Automated Information Systems 


Very Wide 


200,000 


Moving Toward 
Commercial 


Low 


- Commercial Automated Information 
Systems 


Very Wide 


200,000 


Exploits Object 
Oriented 
Technolo^ 


Low 


ReuMbte Compqnent^ Produtts 
- DoD Component Stores 


Very Wide 


50,000 


Tied to Weapon 
System Cycle 


Low to High 
(Depending on Use) 


- Commercial Shrink Wrapped 


Very Wide 


50,000 


Tied to 
Commercial Cycle 


Low 



DoD and commercial applications for software are different in some ways but very similar in 
others. The above table highlights these similarities and differences for real-time applications, command 
and control applications, MIS systems and reusable components and products. This apparent disparity in 
the classification of software between DoD and conmiercial vendors increases the complexity of DoD's 
management task. Companies (people, methods, and organizations) are usually specialized to one or more 
conmiercial type systems, not to DoD type systems. Defense and commercial software applications needs 
are merging, providing the potential that DoD can exploit commercial capabilities more effectively over 
time. Major revisions are needed in DoD software management across all DoD applications in order for 
DoD to capitalize on the evolving commercial base. 
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Process Comparison Summary 



Civilian/ 



Process Attribute 


Custoin 
DoD 


Commercial 


Commercial 
Product 


Commercial 
Result 


Problem Definition 


oWnedbyOH^dHtor 


Shared (Includes 
Market Analysis) 


Miditet Place 


More Practical 
Requirements 


Process Definition 


eySpecHigld 


Intemal/Evoiving 


IntomaVEvoiving 


Reduced Cost/ 
Schedule 


Rexibillty 


VttrytimHMf 


Not Constrained 


EesontiaSy 
Unlimited 


Process Improvement 
Ease, Enabling 
Technologies 


Milestones/Reviews 
(Duration, Formality) 




Short Informal 
incremental 
Document 


, Short, informal 


Reduced Cost/ 
Schedule 


CustomerAJaer 
Involvement In 
Development 


Custoffw tntefwntftm 
\ht6r* Cu«tomtf 


High User at Prob. 
Def. and Acceptance 
User° Customer 




Less (Requirements 
Churning. Reduced 
Cost^hedule 


Process Monitoring by 
Customer 


Hetvy 


Some 


None 


Reduced Cost/ 
Schedule 


Customer Acceptance 
Process 




Simple 


Marketplace 


Reduced Cost/ 
Schedule 


Inspection/Testing 


Rigorous /Formal 


Rigorous/Formal 


Itigorotia/Format 


Similar Product 
Quality 


Subcontracting 


By Spec 


Brief Product Spec 
ISO Possible 


Brief Product Spec 
^0 Pos^bfe 


Quallty/DependabiHty 
of Subcontractor May 
Be Less 


Use of Advanced 
Development Techniques 
(l.e. Reuse/4GU00) 


Implidtiy 
Oiscouragecl 


Extensive 


Extensrve 


Schedule Higher 
Quality 



Source: IBM Federal Systems 



As is evident from the process comparison summary above, there are not only differences between 
commercial and defense applications, but also in the process used by each to develop, acquire and support 
complex software systems. Defense and other federal acquisition regulations and detailed specifications 
require a much more complex set of deliverables within the context of DoD' s contracting process. 
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Comparison ofDoD and Commercial 
Software Acquisition Practices 



m Requirements Definition 

- Commercial 

- More flexible and open between users and supplier 

- Based on strategic plan and usually cost/schedule driven 

- More willing to adjust requirements based on availability of off-the-shelf 
products 

- Evolves capability 

• Vendor Selection 

- Commercial 

- Much more flexible; no requirement for f aimess or to maintain the public trust 

- Encourages vendors to offer best solution, not meet 100% of requirements 

- Accommodate teaming and long-term relationships 



Development Process 

- Commercial 

- More flexible; product improvements anticipated 

- Team approach with bias toward reuse and tailoring of existing systems 

- Multi-year acquisitions not re-justified each year. 



In essence, the Task Force found major differences between DoD and commercial software 
acquisition practices, as outlined above and on the next page. 
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Comparison ofDoD and Commercial 
Software Acquisition Practices (Cont.) 



m Business Practices 

- Commercial practice more flexible with greater incentives 

• Integration Testing and Delivery 

- Commercial provides integration and functional testing according to need 

- DoD uses separate test agency with added time and complexity 

- Absence of beta testing within DOD increases costs 

• Rights in Data 

- Commercial more flexible, especially regarding resales 

• Maintenance 

- Commercial: Maintenance considered and integrated with development 

- DoD: Maintenance not major factor in development process 



Appendix C provides a more complete comparison of DoD and commercial software acquisition 
practices as developed by this Task Force. 
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Assessment of Current DoD 
Software Acquisition Approach 



• Strengths 

- Highly Structured Process Tied to Individual System 
Developments 

- Tightly Defined Requirements 

- Produces High Quality Product for Mission Critical Systems 
That Demand Extremely Low Failure Rates (e.g.. Flight 
Controls for Man-Rated Platforms) 



The strengths of the current DoD approach to software development, acquisition and operation are 
summarized above. The Task Force found that the highly structured DoD process has, in fact, provided a 
high quality software product, in most cases. 
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Assessment of Current DoD 
Software Acquisition Approach 



Weaknesses 

- High Life Cycle Cost in Time and Dollars 

- Software Development Cycle Tied to Weapon System Developments 

» Incredibly Long , Typically 13 to 15 Years from Concept to Fielding 

- Encourages Excessive Acquisition Agent Involvement in Design 
Detail and Process 

- Based on Mistrust vs. Mutual Trust 

» Excessive Documentation 
» Excessive Formal Review 
» Excessive Testing of Non-Critical Systems 

» Poor Communication Between Vendor, Acquisition Agent and User 

- No Requirements Addressing Cost and Schedule 

- Traditional Approach Used: Design it All and Then Build it 

- Little or No Requirements Relaxation for High Cost Items 

- Inadequate Beta Testing in Early Phase 

- Little Focus on Designing in Reusability 



However, there are a number of weaknesses associated with the current defense approach, as 
summarized above. Many of these weaknesses derive from the need for a fair and open procurement 
process and the necessity to prove that public dollars are wisely spent. 
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Assessment of Current Commercial 
Software Acquisition Approach 



m Strengths 

- Open Architecture Compatible with Usage of COTS Software 

- Much Less Formal Competitive Procurement Process 

» Includes Prior Performance as Major Selection Criterion 

- Trend Toward Joint Assumption of Risks Between Buyers and Suppliers 
Across Development, Operation, Maintenance and Modernization 

- Process is More Flexible 

- Shorter Cycle (Product Release) Times 

- Tailorable Level of Documentation and Oversight 

- Emphasis on Reuse and Tailoring Requirements to Existing Products 

- Beta Testing Widely Used 

# Weaknesses 

- Less Responsive to Continual Changes in Requirements 

- Less Assurance that Software Will Function Properly Under All Situations 

- Potentially Locked into One Vendor's Proprietary Application 

V J 



The strengths and weaknesses of the current commercial approach to software development, 
acquisition and operation are summarized above. 
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Principal Reasons DoD Software 
Programs Get Into Trouble 

• Poor Requirements Definition 

- Lack of User Involvement in Development Process 

- Inability of Users to Foresee Benefits of Automation Without Incremental Capability 

• Inadequate Software Process Management and Control by Contractor 

• Lack of Integrated Product Teams 

- Failure to Establish 'Team" With Vendors and Users 

- Little Participation of Functional Area Experts 

• Ineffective Subcontractor Management 

• Lack of Consistent Attention to Software Process 

• Too Little Attention to Software Architecture 

• Poorly Defined and Inadequately Controlled Interfaces Between Computer 
Hardware, Communications and Software 

• Assumption That Software Upgrades Can "Fix" Hardware Deficiencies 
(Without Assessment of Cost and Schedule Risks) 

• Focus on Innovation Rather than Cost and Risk 

• Limited or No Tailoring of Military Specifications Based on Continuing 
Cost-Benefit Evaluations 



Over the last two decades, a number of DoD software development efforts have gotten into trouble, 
particularly in terms of actual costs and schedule vs. expected or predicted costs and schedule. The Task 
Force heard briefings from a number of DoD program managers where this was the case. Based on the 
specific programs discussed and on other inputs, the Task Force prepared a listing of the principal reasons 
DoD software programs have gotten into trouble as shown above. Certain of these reasons are a reflection 
of DoD practice. Others are tied to the software engineering level available at the time such programs were 
initiated. The Task Force believes that today's software technology and practices can directly addresis 
many of the root causes for such past problems. 
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Resource Allocation: 
Where Does the Effort Go? 





Military 


Commercial 


• Engineering 


30% 


50% 


• Evaluation 


20% 


20% 


• Management 


15% 


10% 


• Meeting Support 


15% 


5% 


• Documentation 


15% 


7% 


• CustomerAJser Support 


5% 


8% 



Source: Magnavox and Capers Jones 



There are some indications that commercial development efforts have achieved better predictability 
and lower costs than DoD counterparts. The Task Force solicited quantitative data on this issue from both 
government and conmiercial software experts. The figure above summarizes one type of indicator of the 
difference between commercial and govemnient projects (in terms of the percentage of the effort expended 
for different aspects of a typical development). 
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Comparison of 
Commercial and Government Projects 



Application Type 


Number of Projects 


Average Size (SLOC- 
New and Modified) 


Schedule/Time 
(Months) 


Effort 
(Staff Months) 












Commercial 


1491 


26,000 


12.52 '^'^'^^ 




Government 


75 


26,000 


15.1 




Percent Difference 






+ 17.09% 


+ 44.24% 






; 




Commercial 


259 


26,000 


17.4 


65.7 


Government 


29 


26,000 


20.9 


117.3 


Percent Difference 






+ 16.75% 


+ 43.99% 












Commercial 


295 


212,000 


22 


' 150.2 


Government 


21 


212,000 


25.6 


242.9 


Percent Difference 






+ 14.06% 


+ 38.16% 








; - 




Commercial 




442,000 


45 


1736 


Government* 


7 


442,000 


52 


2740 


Percent Difference 






+13.46% 


+36.64% 


Summary of 

Average Percent Difference 






+15.34% 


+40.76% 



* Small sample may be statistically invalid. 
Source: QSM, Inc. 



This figures summarizes other quantitative indicators of the difference between commercial and 
government projects (in terms of the typical size of the code, time to develop the application and overall 
level of effort). 

It should be noted that the Task Force was unable to find rehable, quantitative data supporting the 
notion that commercial practices are more cost-effective than DoD practices. This lack of reliable 
indicators was a major concern of the Task Force. 



3.0 MA.TOR FINDINGS AND RECOMMENDATIONS 

3.1 PROCESS CREDIBILITY 



Process Credibility 



Findings 

• Attempts to achieve absolute fairness in competition in contracting 
(fair and reasonable pricing) have led to a lack of trust between 
government and individuals/contractors 

- Current DoD practice: 

» Is not compatible with commercial business practices 

» Is focused on contractor management/audit activities and costs 

» Hinders flexibility (managers take no personal risk) 

» Is not well suited to procuring complex, knowledge-intense, "first of a kind" systems 
» Is too costly 

» Does not prevent malfeasance or incompetence 
» Leads to adversarial relationships 

» Reduces reuse and contractor incentive to invest because of stringent data rights 
interpretation 

- Contractors must make profit on single contract; no long term relationship (DoD 
business practices do not view profit as a legitimate cost of doing business) 

- No individual fully understands or owns total process 



The Task Force spent considerable effort on how the requirement for DoD to ensure public trust in 
its acquisition process influences DoD's ability to employ "commercial best practices." The Task Force 
found that attempts to achieve absolute fairness in competition in contracting have, in fact, led to a lack of 
trust between the government and individuals/contractors. DoD acquisition processes are focused on 
contractor audit activities to an extent that hinders the flexibility of Program Managers and contractors. 
DoD PMs are not incentivized to assume any personal risk associated with allowing for flexibility 
comparable to commercial practice. Criminal sanctions are a significant disincentive for such efforts. 

DoD's acquisition system still does not prevent malfeasance or incompetence and leads to 
adversarial relationships rather than partnerships which are the norm within commercial industry software 
developments. One problem highlighted by the Task Force was that the current DoD system does not 
allow one individual or manager to control the total process, even for a specific project. This lack of 
control leads to a diffusion of accountability and hinders DoD's ability to oversee complex "first of a 
kind" software developments. 
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Process Credibility 



Finding s 

• Requirements and Source Selection Inflexibility 

- Vendors attempt to meet eveiy requirement 

- Ease of protest 

- Requirements focus vendor on particular solution 

- Requirements have become alternative to professional judgment 

- Departures from requirements have caused protests and bad publicity 

• Price/Schedule/Functionality 

- In commercial sector, managers constrain 2 out of 3 (e.g., cost and schedule) 

- In DoD, managers constrain 3 out of 3 

• Constrained Communication During Solicitation 

- Questions are provided to competitors (can give away proprietaiy concepts) 

- Questions are often misinterpreted and answered incorrectly 

- Guarded way of asking questions limits substantive feedback 

• Complicated Regulations 

- Restrict variety of proposals 

- Restrict competition and limit government options 

- Consume time and resources 

- Drive government employees to follow conservative procedures as safest path (inhibits "best value") 

• Government-Unique Accounting Procedures and Audits 

- Audit requirements limit available vendors 

- Add substantial cost 

- Create need for separate accounting systems 

- Drive vendor to lowest labor cost solution rather than best value solution 

- Lead to rejection of good solutions based on value pricing vs cost-based pricing 



The viewgraph above lists important hindrances that the Task Force sees with regard to the 
adoption of commercial software practices. Many of the Task Force recommendations address these 
hindrances. 
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Process Credibility 



Recommendations 

• Make necessary changes to acquisition regulations 

- Have Program Managers Manage 3 of 3 (Price/Schedule/Functionality) But Only 
Constrain 2 of 3 

- Define Successful Performance on Contracts as Delivering Solution (with Predictable 
Price, Schedule and Functionality) Not Adherence to Government Processes, 
Procedures and Specifications 

- Do Not Require C-Level Specifications for Software Projects Developed in Ada 

- Prior to RFP, Government Should Perform Independent Market Analyses of Off-the- 
Shelf and Contractor Products to Assure "Best Value" Solution 

- Establish Mechanisms to Allow Both Current Ability to Perform as Well as Past 
Performance as Key Factors in Source Selection 

» Require Source Selection Evaluation of Development Contractors Through a Formal Software 
Process Capability Evaluation 

- Encourage Offerors to Demonstrate as Much Functionality as Possible as Part of Bid 
Without Eliminating Domain Knowledgeable Competition 

» Executable Architecture as a Minimum 
» Weight Heavily in Selection 



The Task Force makes the above recommendations with regard to process credibiUty. 
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3.2 DOD PROGRAM MANAGEMENT 



DoD Program Management 



Findings 

• Program management does not encourage "80% solution for 20% cost" 

• Users often not significantly involved in process 

• Little emphasis on life-cycle issues (including maintenance and support) 

• Existing policies, methodologies, and procedures, and their implementation are 
inadequate 

- Little evidence that policies are influenced by actual experience and vice versa 

- Little effort to measure effectiveness and costs of policy directives 

• Some indication that DoD is migrating toward development and use of 
standards-based architectures 

• Program Managers lack incentives to allow tailoring of procedures to fit 
individual program needs or to develop "corporate" solution (e.g., employ 
common architecture or common software components) 



The Task Force found that the DoD program management system itself discourages the use of 
conmiercial best practices. These findings are not unique to software, but are particularly important for 
software given the significant cost savings associated with software reuse. There are few incentives for 
Program Managers (PMs) to develop or even employ corporate solutions (common architectures/software 
components), particularly if they are more expensive to acquire. Rather, each PM will tend to optimize 
his/her development for hs own purpose. Further, it is difficult for users to be significantly involved until 
late in a software development process, unless some sort of prototype can be constructed. DoD PMs place 
little emphasis on life-cycle issues (such as software maintenance and support). Existing DoD-wide 
software policies, methodologies, and procedures, and their implementation by PMs are inadequate. 
There is little evidence that policies are influenced by actual experience and vice versa and there is little 
effort to measure the effectiveness and costs of policy directives. 
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DoD Program Management (cont.) 



Recommendations 

• Establish Overarching Software Life Cycle Guidelines 
- Tools/Methods 



>» Define Software Architectures to Enable Rapid Changes and Reuse 

» To Achieve the Benefits of Using Standards-Based Architectures, DoD Must Manage Programs Using: 

• Early system engineering 

• Iterative development 

• Proactive participation in development of these standards 

» Promote Development/Use of Community-Wide Metrics and Models (e.g., SEI's Capability Maturity Model) 



» Revise the Milestones for Software-Intensive Development 

• Address the need for a software-first philosophy 

• Provide for a layered softwaie/hardware standards based architecture 

• AcquisiHoin and life cycle planning should now separate hardware and software fieldings based on the business sense of the specific project 
» Require Early Interaction Between User, Acquisition Agent and Developer; Identify and Get Early User Involvement 

» Apply Evolutionary Development with Rapid Deployment of Initial Functional Capability 
» Encourage Competition of Technical Approach vs. Cost 

» Provide Incentives and Guidelines to Encourage Software Reuse (Architecture-Based Reuse) 

» Reduce Documentation and R^ew Requirements for "Mature" Companies (i.e.. Companies Determined to Be "Mature'' 

Through Evaluation Mechanisms) 
» Tailor operational testing to develop DoD "Beta Teaif philosophy 

• Allow fielding of software direct from test beds with user consent 

» Have Program Maiuger Stay with Programs at Least Through Beta Testing to Maintain Continuity of Understanding of 
Original Nuances in Requirements 



- Acquisition 





The Task Force makes the recommendation that DoD establish overarching software lifecycle 
guidelines directed at facilitating program manager employment of commercial practices and software and 
that a DoD-wide effort be made to oversee implementation of these guidelines. The tools, methods and 
acquisition approaches recommended by the Task Force are listed above. 
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3.3 DOD PERSONNEL 



DoD Personnel 



Finding s 

• A shortage of sufficiently qualified software personnel 
currently exists at all levels within the DoD 

- Expertise for software acquisitions, software evaluations, and software 
maintenance/support 

- Expertise to represent DoD (customer) interests with commercial sector 

- Expertise in domain software design and applications 

- Expertise in software technology to develop policies, standards, and 
guidelines 

- Expertise in software program management 

V / 

Based on the inputs provided, the Task Force found that a shortage of sufficiently qualified 
software personnel exists at all levels within the DoD. Without personnel who are highly qualified in 
modem software practices, DoD will not be as capable of effectively exploiting complex software within 
its systems. This personnel shortage has been a major contributor to the problems that have arisen in past 
DoD software development programs. 
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DoD Personnel (Cont.) 



Recommendations 

• Establish DoD-wide software program management education and training 
initiative 

- Change DSMC and IRMC courses for FMs to reflect best commercial practices and other 
recommendations of this Task Force and Provide for changes to reflect the dynamics of the 
software industry 

- Develop and provide interactive training tools for senior managers to perfect software 
management skills 

- Rotate government and contractor personnel between FM and developer organization to build 
understanding and trust; encourage use of IPA's from industry 

- Incorporate software management principles in senior management education and seminars 
(including senior services colleges) 

- Frovide mechanisms for keeping software expertise current in the workplace 

• Develop Acquisition Managers with software program management expertise 

- Integrate software-qualified personnel into senior acquisition staff 

• Establish Norms for the Number of Software Experts in Program Offices 

• Upgrade Educational Requirement for Personnel Assigned to Acquisition, 
Management, Development and Oversight of Software Intensive Programs 

Develop Expertise in Analysis of Domain Software Design 

- Fromote Software Reuse in the Design 



The Task Force makes the above recommendations with regard to DoD software expertise of its 
personnel. The Task Force strongly recommends an emphasis on increasing the capabiUty of its personnel 
in modem software practices and techniques. 
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3.4 USE AND INTEGRATION OF COMMERCIAL OFF-THE-SHELF SOFTWARE 

Use and Integration of 
Commercial Off-the-Shelf Software 



Finding s: 

• DoD Does Not Normally Perform COTS Market Analyses Incident to 
Requirements Definition 

• DoD is Beginning to Exploit Evolutionary Development Approaches 

• Tools/Methodologies Are Evolving to Facilitate Use of COTS 

- Computer Aided Software Engineering (CASE) 

- Object oriented methods 

- Reuse repositories 

- Integrated product teams 

- Integrated risk management 



V y 

DoD does not routinely perform COTS software market analyses during the requirements 
definition phase of an acquisition program. Nor does it employ prototypes or simulations of new 
capability in a way that could influence the requirement process. Rather, requirements are evolved in a 
manner that is disconnected from the availability of commercial applications and are not typically 
influenced by such available capability. This situation is true for both hardware and software. Today's 
technology can facilitate early interaction between users and available capability, particularly in software. 
Further, DoD is beginning to exploit evolutionary software development approaches that could provide for 
the inclusion of existing conmiercial software fimctionality into early prototypes and allow users to test 
such capabilities. Modem software tools and methodologies have evolved that facilitate use of COTS, 
such as those listed above. 
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Use and Integration of 
Commercial Off-the-Shelf Software 

Findings (Cont>) 

• DoD has not fully identified the pros and cons associated with the use of COTS 

- Pros 

» Saves money in design and development of those components 
» Can Significantly Reduce Development Risk 

• Support is Concern 

» Selection of COTS products early in project cycle will enable requirements to be driven by commercial software 

capabilities 
» Very useful in rapid prototyping 

- Cons 

» Commercial products must still be integrated into system and qualified with system 

» Difficulty in Configurement Nfanagement and Support for Older Releases 

» Additional testing may be required to qualify system with commercial components 

» Subsequent releases determined by vendor, not DoD 

» Security Aspects of Use of COTS Not Well Understood 

• DoD not addressing multiple aspects 

- Development environment 

- Tactical computer program 

- Viftts protection 

- Commercial Espionage 

• Qassified software systems a problem for commercial companies 

• In addition, DoD has not detennined when to use COTS 

- Commercial Off-the-Shelf Software Products May Not Apply to All DoD Systems 

- Should be used as is - avoid tailoring or special features 

- Most weapon system/real-time application software for DoD will not exclusively be "custom," but 
will involve some COTS 



In general, DoD has not identified the pros and cons associated with the use of COTS. DoD must 
learn how to balance the cost savings associated with design and development of commercial software 
products and the significantly reduced development risk with the concern for longer term support and 
system security. Commercial products must still be integrated into and qualified within each defense 
system and there is difficulty in configuration management and support for older releases. The latter point 
is important since many DoD software systems are not deployed before a conmiercial software product is 
retired or replaced. DoD is not addressing many aspects of the use of COTS software: 

- Development environments 

- Virus protection 

- Commercial espionage 

In essence, DoD has not developed a corporate way to decide when to use COTS software. 
Conmiercial off-the-shelf software products do not apply to all DoD systems. Most weapon system/real- 
time application software for DoD will not exclusively be "off-the-shelf." Integration and configuration 
control for COTS software then become important concerns. Further, DoD must learn to use COTS so 
that it avoids tailoring or special features. 
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Use and Integration of 
Commercial Off-theShelf Software 



Recommendations 

• Require Trade Studies and Analysis of the Use of COTS in DoD's Software Acquisition 
Process Where Effective 



- Performed by Acquiring Organization as Essential Part of Defining Requirements and in Rapid Prototjrping 
Situations 

» Employ Broad Agency Announcemento or Similar Contractual Approaches to Facilitate Such Studies 

- Use of COTS Appropriate When: 

» Defining Requirements 

» Rapid Prototyping Situations 

» Not Required to Tailor COTS Source Code to Application 

» Not Required to be Error-Free 

» COTS Software is "Qose Enough' to Tailor Requirements 



• Establish "Customer Friendly" Application-Specific Information Technology 
"Component Stores" 



- Generic Architectures for Specific Domains 

- Rapid Requirements Definition Process and Prototyping 

- Reusable, Prequalified Components 

- Assemble Systems Rather than Develop Them 
Reduce Lead Time 

- Security is not Paramount 



• Increase tech base funding for security audit tools for systems employing COTS 

• Capitalize on Innovative Cost-Effective Techniques for Acquiring and Using COTS 
Software Products 

- Such as Use of Enterprise Licenses 





Given these findings, the Task Force makes the above recommendations with regard to DoD use 
and integration of conmiercial off-the-shelf software. The Task Force sees great benefit to be gained 
through exploitation of COTS software; however, DoD must develop corporate approaches to the use and 
integration of COTS software, if it is to gain this benefit. 
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3.5 ACQUISITION 




Acquisition 




• Finding s 

- Acquisition practices have led to: 
» Distance between user and developer 
» Limited participation by commercial software companies 



- Adherence to DoD regulations for reviews and documentation is increasing 
software costs 

» DoD software costs are estimated to be increased by at least 20 % over 
commercial best practice 

» Commercial best practice requires much less documentation than DoD 

- Funding for maintenance planning/execution starts late 

» Maintenance assumed organic; inhibits teaming/partnerships 

- Acquisition focus is on mandatory "how to" specifications and standards 
rather than the product (what) 

» Lengthens process and adds costs 

» Discourages harmonization with commercial practice 

» Creates adversarial relationship 

- Acquisition process does not reward development of reusable software 



The Task Force was concerned that the specific acquisition and contracting approaches used by 
DoD inhibit use of commercial practices and software. The Task Force findings in this area are shown 
above. Strict adherence to DoD regulations for reviews and documentation is increasing software costs. 
DoD software costs are estimated to be increased by at least 20% over commercial best practices. The 
DoD focus on detailed technical specifications has lengthened the process, added costs, discouraged 
harmonization with commercial practice, and created a highly adversarial relationship between the 
Government and industry. There is a strong belief that certain commercial companies or divisions of 
companies have opted to stay away from government contracts due to the complexity of the acquisition 
rules and regulations. 
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Recommendations-Acquisition 



Recommendations 

- Implement the following acquisition approach: 

» Establish acquisition focus on functionality and consistency with 
"commercial best practice" 

» Revise procedures encouraging interaction between user and developer 
and achieving early functionality 

» Minimize DoD regulations for review and documentation that are 
different than "commercial best practice" 

» Require planning for maintenance at beginning of development process 

» Provide govemment funded vehicle in contracts to incentivize 
development of reusable S W 

- Review all existing military standards and military specifications 
pertaining to software development and documentation, for 
continued applicability, such as DoD-STD 2167 



To this end, the Task Force makes the above recommendations with regard to DoD software 
acquisition practices. In essence, these recommendations can move DoD toward an acquisition approach 
that is more consistent with conmiercial approaches, while not requiring changes in statutes. 
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Proposed Software Life Cycle 



Pre-Prososal Demos & 
Market Analysis 




Proposal & Demo 
Simulation Capability 



Version 1.0 
Basic Functions 



Version 2.0 
Improved Functions 



Version 3.0 
Improved Functions 



To implement this recommendation, the Task Force proposed that DoD evolve a more appropriate 
software life cycle approach as depicted above. This approach provides for early capability (even in the 
initial bidding process) and for a gradual enhancement in capabiUty over time. 
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Acquisition Approach 



Developing the RFP 




Integrated Top Level Specification 

# Perfonnance 

# £nvuroxtment 

# TestA^alidation 



' 



Selection 

• Acquiring Agency 

^ Evaluate Technical Solution aj\d Management Flan 
: ~ Evaluate SW Process Maturity and Past Peifottiahce [ 

# Users 

_ Define Requirements 
^ Evaluate Demonstrations 
^ Heavy Farticipatton in Seleclion 




Contract Proposal 

RFP Response 

Qualified Software Oi^nization . 



Domain Experience 
Skills Matrix 

Development Environment 
Process Control 
Feer Reviews 

Metrics and Milestone Plan 
Reuse Plan 

Program Manag;ment Flan 



Proposal Demonstration 




The Task Force proposed approach to competition is shown in the above figure. The core 
government role in such an approach would be to: 

• Develop the RFP (no "How To" statement of work; no management CDRLs; no govemment in- 
process approvals) 

• Provide an integrated "Top Level" specification (architecture, COTS/reuse, software engineering 
environment and test/validation approach). 

• Employ software metrics as a key determinant, 

• Evaluate proposed technical solutions and the proposed management plan 

The contractor would then: 

• Provide an execution plan, management controls and progress milestones/metrics 

• Describe an in-place, mature software development organization and relevant domain experience 

• Provide a skills matrix describing personnel to be employed 

• Identify a robust development environment and describe applicable prior experience 

• Describe automated process control software 

• Describe the extent to which peer inspections will be used 

• Provide a metrics usage plan and purposes for which they will be used 

• Provide specific reuse and program management plans 

• Propose specific architecture(s) in executable code 
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3.6 SOFTWARE ARCHITECTURE 



Software System Architecture: 
The Missing Link? 

What is software architecture? 

• Software architecture consists of: 

- Software system components 

- The relationships among those components 

- Rules for their composition (constraints) 

m Defining document would address: 

- System functionality 

- Software system components 

- Interfaces, standards, protocols 

- Execution model 

- Dataflows 

- Control flow 

-> Critical timing/throughput aspects 

- Error handling 



Software architecture was emphasized by the Task Force as a means for achieving important ends. 
Software architecture consists of software system components, the relationships among these components 
and the rules for their composition (constraints). To use architecture as a management tool, DoD needs to 
define: system functionality along with software system components to be employed; interfaces, 
standards, and protocols to be employed; and the execution model to include data flows, control flow, 
critical timing/throughput aspects, and error handling approach. 
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Software System Architecture (Cont.) 



Findings 

• Why is it important? 

- Essential for effective management over the lif ecycle 

» Software system lifecyde costs - 65% for maintenance 

» - 65% of maintenance costs are due to changes/modifications/upgrades 

- Software architecture is a prime enabler of flexibility and reuse 

- Well-formulated architecture might reduce costs of changes/upgrades by 30- 
50% ($4-$7B/year assuming software expense ^ $30B/year) 

• Why doesn't it play a larger role? 

- Focus is usually on initial cost schedule, functionality - not lif ecycle 

- 2167A reinforces this approach - requires proof that design satisfies 
functionality 

- Difficult to specify, test, etc. 

- Not well understood 



Software architecture is a prime enabler of flexibility and reuse, and a well-formulated architecture 
might reduce costs of changes/upgrades by 30-50% ($4-$7B/year assuming software expense 
~$30B/year). Software architecture has not been emphasized because PM focus has usually been on initial 
cost, schedule, and functionality and not on the life cycle. DoD-STD-2167A reinforces this approach by 
requiring only proof that a design satisfies the required functionality. 
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Software System Architecture (Cont.) 



Findings (Cont.) 

• Little emphasis on architecture in DoD software programs or regulations 

• Impact of software architecture issue 

- DoD not benefitting from architecture as a key tool for: 

» Evolutionaiy development 

» Early (and often) involvement of users with functional capability 
» Ability to include changing commercial technology 
» Reuse 

» Facilitating requirements change with minimum cost and schedule 
» Facilitating product line management 

• Insufficient Progress in: 

- Developing models/standards for domain specific software architectures 

- Open system architectures work 

V J 



There is currently little emphasis on architecture in DoD directives or regulations. As a result, DoD 
is not benefiting from architecture as a management tool. Further, the Task Force sees insufficient 
progress in developing models and standards for domain specific software architectures. 
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Software System Architecture (Cont.) 



Recommendations 

• Emphasize Use of Software Architecture 

- Establish model and context for architecture selection 

» Standards-based with emphasis on "implementable" 

» Require vendors to propose, manage and control the architecture 

- Require delivery of software architecture definition as first step in any 
software acquisition 

- Foster migration strategies at architecture level 

• Assign responsibility within Government for domain analysis and product 
line developments 

• Provide expertise and resources to ensure coordinated DoD participation 
in commercial/international standards bodies and users groups 



The Task Force makes the above recommendations in order to facilitate greater use of software 
architecture as a management tool for DoD software programs and activities. In particular, the Task Force 
sees great value in requiring the delivery of software architecture definition as a first step in any software 
acquisition. Where possible, such software architecture definition should be operational (i.e., executable). 
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3.7 SOFTWARE TECHNOLOGY BASE 



r 



Software Technology Base 




Findings 

• The current DoD software technology base does not 
adequately take advantage of commercial R&D 

• Software technology transfer ( internal and external) is not 
receiving adequate emphasis within DoD 

• There is a paucity of data to support prediction of cost, 
schedule and performance 



The software technology base (both defense and commercial) provides DoD with ample 
opportunity for significantly improving the defense software acquisition life-cycle. 
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Software Technology Base (Cont.) 



Recommendations 

• Provide for the evolution of the DoD Software Technology Strategy to align 
with emerging commercial technology and practices 

• Emphasize Technology Transfer (External and Internal) 

- Fund technology transfer programs in such topics as: 

» Architectural principles, 

» Architecture description languages, 

» Standard interfaces, and 

» Integration technologies 

- Initiate demonstration programs (e.g., ATDs) to facilitate software technology 
insertion into systems. Examples of candidate criteria: 

» Open standards 

» Use of COTS and GOTS 

» Frequent releases to include a number of users 
» Multiple platforms 

» Satisfies commercial standards and interoperability standards across DoD 
organizations 

Initiate formal data collection and analysis 



The Task Force makes the above recommendations with regard to DoD software technology base 
investments. The Task Force supports a DoD technology base program that is more closely aligned with 
the wide range of similar efforts ongoing in commercial organizations. 
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4,0 



SUMMARY 



4.1 



"ATTA PERSONS 




'Atta Persons 




m Electronic Systems Command Software by Assembly of Modules 

- Software "Component Store" 

• A-109 Acquisition Methodology 

• Use of Commercial Practices and Products in Reserve 
Component Automation System (RCAS ) 

• Software Reuse, Prototypes and User Involvement of Global 
Command and Control System (GCCS) Initiative 

• Perry-Paige Initiative Toward Migration Systems 



The Task Force identified several ongoing efforts worthy of note. 

• The Air Force Electronic Systems Command (ESC) has defined an approach for software system 
development through the assembly of pre-qualified software modules (PRISM). ESC is pursuing 
the development of such software modules. The Task Force was very supportive of this program. 

• Office of Management and Budget (OMB) Circular A- 109 outUnes an approach for major Federal 
system acquisitions that encourages: definition of top level needs vs. detailed specifications; 
exploitation of innovative private sector contributions and use of early competitive demonstrations 
of competing approaches. These all are acquisition attributes recommended by the Task Force. 

• The Reserve Component Automation System (RCAS) is an ongoing MIS development to support 
the reserves. The program has successfully employed the A-109 acquisition approach and 
extensively used commercial acquisition practices and products. The Task Force conmiends this 
approach. 

• The Global Command and Control System (GCCS) is an initiative of the Joint Staff (J3 and J6) to 
provide vertical and horizontal interoperability of combat information systems across Services, 
Combatant Conmiands and Agencies. It was highlighted for its software reuse approach, its use of 
prototypes and its emphasis on operational user involvement. 

• The Perry-Paige migration systems initiative has established a focus on selecting a set of target 
computing systems (including MIS, C3I and embedded) towards which DoD will aim. This 
migration strategy will enable a more cost-effective DoD investment in software across the life- 
cycle. 





4.2 OVERARCHING RECOMMENDATION 



Overarching Recommendation 



SECDEF Assign USD (A&T) the Responsibility for DoD-Wide Software 
Technology, Policy, Practices and Acquisition 

This Responsibility Includes Allocating Resources and Assigning 
Responsibility for: 

- Drafting and Institutionalizing a New Software Acquisition Policy That 
Includes the Recommendations of this DSB Task Force 

- Creating Incentives and Ensuring Compliance to the Policy 

- Creating Terms and Conditions and Financial Rewards to Maximize the 
Ability of DoD to Realize the Benefits of Conunercial Best Practices 

' Requiring Source Selection Evaluation of Development Contractors Through 
a Formal Software Process Capability Evaluation 

- Advising the Acquisition Executive on Matters Conceming Software 
Technology, Acquisition and Architecture for Major Programs 

- Maintaining a Digest of Lessons Learned and Best Practices and 
Communicating the Same to Program Managers and Contractors 



Throughout its deliberations, the Task Force acknowledged that the issues associated with defense 
software were applicable across the spectrum of DoD software intensive systems. The Task Force also 
frequently learned of obstacles based, in part, on the current dichotomous DoD structure associated with 
software technology, policy, practices and acquisition. In order to ensure that the DoD reap maximum 
benefit from its recommendations, the Task Force formulated the above overarching recommendation. 
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Overarching Recommendation (Cont,) 



• In Carrying Out This Responsibility, Consider Forming an Executive 
Council 

- Including the DDR&E, the ASD (C3I) and Appropriate Representatives from 
the Services and Defense Agencies 

- Provide Supporting "Process Action Team" to Assist in Implementation 



The Task Force also identified the mechanism by which its overarching recommendation could be 
readily implemented. 




APPENDIX A 
TERMS OF REFERENCE 



THE UNDER SECRETARY OF DEFENSE 

WASHINGTON. DC 20301-3000 



OtO 0 6' 1993 



MEMORANDUM FOR CHAIRMAN, DEFENSE SCIENCE BOARD 

SUBJECT: Terms of Reference — Defense Science Board Task Force on 
Acquiring Defense Software Commercially 



You are requested to form a Defense Science Board Task Force 
on Acquiring Defense Software Commercially. Determine the 
conditions under which the procurement of defense software (i.e., 
operational software, support software, and software tools) can 
appropriately use commercial practices and develop a strategy for 
defense software procurement that substantially incorporates such 
practices. Specifically address within this strategy DoD use of 
commercial software products. 

The scope of this effort should include all DoD systems that 
are software intensive. It should address all stages in the life 
cycle of a software component from initial procurement to 
evolutionary upgrade of software or of software/hardware 
combinations. This commercially-based strategy should not be 
constrained by existing DoD standards; it should be viewed as a 
coexisting alternative to, rather than replacement of, the 
current DoD procurement strategy. Accordingly, you should 
compare your proposed strategy to the current DoD strategy to 
indicate circumstances in which e$ich strategy is most beneficial. 

To assess the appropriateness of DoD use of commercial 
practices, the Task Force should identify and apply objective 
measures such as elapsed development time, software life-cycle 
cost, management risk, as well as measures of software product 
quality. 

The Task Force should consider at least the following 
topics: 

Technical: state-of-the-art and best commercial 
practices, development tools, reusable software components, 
and techniques and tools for tailoring commercial software 
components for use in defense systems. 

Management: DoD management of the commercial development 
process, software risk management techniques and supporting 
tools, minimum delivery time, af f ordability , maintenance 
after product delivery, post-deployment product enhancement, 
software process support tools, quality and assured 
availability, and use of development and maintenance tools. 




Contracting: technical data rights, intellectual 
property rights, liability, alternative forms of procurement 
agreements, and incentives for creation of reusable software 
components as well as their subsequent reuse. 

The Director, Defense Research and Engineering will sponsor 
this study • Dr. George H. Heilmeier and Dr. Larry E. Druffel 
will serve as Co-Chairmen. The Office of the Director, Defense 
Research and Engineering will provide the necessary funding and 
support contractor arrangements. The Executive Secretary will be 
Ms. Virginia L. Castor. Commander Robert C. Hardee will be the 
Defense Science Board Secretariat representative. It is not 
anticipated that this Task Force will need to go into any 
"particular matters'* within the meaning of Section 208 of Title 
18, U.S. Code, nor will it cause any member to be placed in the 
position of acting as a procurement official. 




,John M. Deutch 
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BRIEFINGS 
PROVIDED TO THE TASK FORCE 



Briefings Provided to the Task Force 



Joint Staff Views on How DoD Can Adopt Commercial Software 

Practices 


MG David Kelley, Vice Director J6 


Army Views on How DoD Can Adopt Commercial Software Practice 


LTG Peter Kind, Director, Information Systems for 


Naval Views on How DoD Can Adopt Commercial Software Practice 


RADM John Hekman, Commander, NISMC 


Air Force Views on How DoD Can Adopt Commercial Software 
Practices 


Mr. Lloyd Mosemann, Deputy Assistant Secretary of Air 
Force (Communication, Computers & Support Systems) 


DISA Views on How DoD Can Adopt Commercial Software Practices 


Dr. Mark Scher, Director of Infrastructures, Defense 
Information Systems Agency 


DoD 5000 Series Regulations 


Mr. Gene Porter, Director, Acquisition Program Integration 


DoD 8000 Series Regulations 


Mr. Harry Pontius, Director, C^I Policy 


DoD-STD-2167A/MIL-STD.498 


Dr. Singh, Senior Manager, Space & Naval Warfare Systems 
Command 


Draft White Paper on Software Acquisition 


Dr. Jack Ferguson, Program Manager, Software Engineering 
Institute 


Data Modeling vs. Data Standards 


LTG Peter Kind, Director, Information Systems for C^ 


Case Study: B2 


Mr. Fred Schwartz, Director of Engineering, 


Case Study: Ensemble of Real-time Software Systems 


MajGen Israel, Director, Defense Airborne Reconnaissance 
Office 


Case Study: Reserve Component Automation System (RCAS) 


MG Gary Stemley, Program Manager for RCAS 


Case Study: AEGIS 


CAPT. Richard Cassidy, AEGIS Technical Director 


Case Study: PRISM 


Mr. Robert Kent, ESC/ENS 


Case Study: BMC3 


Lt Col Robert Phelps, BMDO 


Boeing Views on DoD vs. Commercial Software Practices 


Mr. John Hanson, Director, Systems Software & Engineering, 
Boeing 


McDonnell Douglas Views on DoD vs. Commercial Software Practices 


Mr. L. George Hite, Senior Manager, McDonnell Douglas 


IBM Views on DoD vs. Commercial Software Practices 


Mr. Dave Coyer, Program Manager, IBM Federal Systems 
Company 


Object Technology and a COTS Product Called "SNAP" 


Mr. Joe Fox, Chairman, Template Software Inc. 


Procurement Regulations/Issues 


Mrs. Eleanor Spector, Director, Defense Procurement 
OUSD(A&T) 


Integrated Computer Aided Software Engineering (ICASE) Program 


Col Gary Case, ICASE System Program Director 


Computer Sciences Corporation Case Study on Commercial Software 
Development 


Mr. Daniel Kemp, Senior Partner, Computer Sciences Corp. 


Commercial Software Acquisition Practices 


Mr. Alex Morrow, General Manager, Lotus Development 
Corp. 


CApcnCIlCCo 111 UUU oUlLWdlC iVldlllLCIIallL/C 


IVII . DiJlJUy IVlC-L/UllalU, L/CpULy L^llCL/LUl, OiCCLIUlllC WailalC 

Directorate, Warner Robins ALC 


MITRE Corporation Experiences in Exploiting Commercial Practices 


Mr. Steven Crisp, Dept. Head, MITRE Corp. 


rr\r\ OlLlUy Ull lllC UaC Ul Iw^UlIlillClCIal OUllWalC 


iVll . JUllll OlCilUlL, Vice rlColViwlll CX. VJCllvlal iviailagci , 1 fx w 

Systems Integration Group 


i>iOi/\ oiuuy on iiic use ui ^v^iij •juiiwaic 


^rpn \X/illiiim l?ipViJtrrlcon lT^A^13**t^ pYPPiitiv^* Pr**ciH<*tit 

VJdI. W llllalll XxlL>lJalU&Ull, UOAV^IVd^, JJtACCULlVC V IL'C i lColU\^lll 

Army Programs, Burdeshaw Associates, Ltd. and Ms. Linda 
Connor, Program Manager, Rockwell International 


DoD Software Reuse Initiative 


Ms. Linda Brown, ODASD(IM)/Information Technology 


MITRE Study on Feasibility of Using a Software Acquisition Maturity 
Model in DoD 


Ms. Judith Clapp. Division Assitant, The MITRE Corp. 


Software Management Metrics and Reliability 


Mr. Douglas Putnam and Mr. Lawrence Putnam, Jr., 
Quantitative Software Management, Inc. 


Innovative Techniques in DoD SW Acquisition: F-22 Integrated 
Product Team Approach 


Colonel Robert Kayuha 


Legal Impediments to Acquiring Defense Software Commercially 


Mr. Robert Gorman, OSD General Counsel 
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APPENDIX C 



COMPARISON OF SOFTWARE 
ACQUISITION METHODS 



Comparison of Software Acquisition Methodsl 



Requiremen 


ts Definition 


Best Commercial Practice 


Current DoD Practice 


Requirements based on strategic plan and market 
analysis. 


Requirements based on using command Mission 
Need Statement, master plans, and top-level 
certification. 


Requirements based on life-cycle resource 
constraints. 


Requirements based largely on annual budget 
resource constraints. 


Detailed requirements generated by interdisciplinary 
team including users, domain experts, and system 
engineers. 


Detailed requirements generated by buyer in 
collaboration with user. Team generally includes 
domain experts and acquisition personnel. 


Buyer, user, and vendor are a team. Attitude of 
partnership, trust and cooperation. Presumption of 
trustworthiness for reputable commercial 
organizations. 


"Us vs. them" mentality about contractors. 
Government thinks in terms of control, 
accountability, detailed auditing, and double 
checking. Presumption that contractors cannot be 
trusted. 


Functional specification is modified by knowledge 
of availability of existing products. 


Functional anchor performance specif ication; little to 
no regard for existing products. 


Vendors involved early in study, analysis and 
prototyping with emphasis on reuse and evolution 
of existing systems. 


May contract for prototypes, but contractor 
involvement in pre-award discussions is 
discouraged. 


Need is based on business case and decisions are 
based on return on investment, 
("time to make") 


Need based on Mission Need Statement; decisions 
based on need, economics, and politics, ("time to 
field") 


Efficient decision processes. 


Decision processes formal and time-consuming. 


Level of documentation is negotiable based on 
individual user needs and complexity of system 
being developed. 


Extensive (often redundant or unnecessary) 
documentation required under 2167A. 
Tailoring of documentation requirements is often 
minimal or discouraged. 


More detailed analysis of cost versus feature. 
Dropping lower value/higher cost options or 
reducing requirements is practiced. 


Little or no requirements reductions on high cost 
items. 


More requirements trade-off decisions (involving 
complexity and schedule) for reduced time to field. 


Very little flexibility to trade-off requirements creep 
versus complexity and schedule. 


Selected vendors may assist in preparing 
specifications. 


Vendors not involved in preparing specifications. 


Tools used to create system models for use in 
requirements definition; e.g., GUI Building. 


Requirements definition based on Mission Need 
Statement. 


Flexibility allowed in choice of programming 
language. 


Specific requirements regarding use of 
programming language; e.g.,, CMS-2, etc. 


Evolutionary and incremental approach favored. 


Requirements defined up front with little flexibility 
for modifications. 


Summary 

Commercial is more flexible and open between users and suppliers, and requirements are based on a 
strategic plan. In the commercial world, there is more willingness to adjust requirements based on 
availability of products and thus to filed a system sooner and evolve it to include more capability at 
significant cost savings. 



^ This appendix was initially derived from a White Paper on software acquisition methods prepared by the Software 
Engineering Institute. The resulting content represents the consensus of this Task Force. 
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Vendor Selection 


Best Commercial Practice 


Current DoD Practice 


Solicit multiple (but not all) qualified vendors -a 
selected few. Encourage teaming with a view to 
attaining a long term relationship that covers the 
entire life cycle and fosters trade-offs in cost and 
schedule. 


Solicit all possible vendors. Vendor proposals must 
meet 100% of requirements. Teaming seldom 
encouraged; development and maintenance usually 
separate entities. 


Compare vendor history and experience. Maintain 
long-term relationships. 


Can compare previous performance, but normally 
can*t have long-term relationships. 


The organization that will be responsible for a 
system over its full life cycle is heavily involved 
from the beginning. 


Maintenance organization not usually involved in 
vendor selection process. 


Use site visits and demonstrations to gain 
knowledge of vendor capabilities. 


Site visit only by capability evaluation team, or 
other expert teams. Visits are very structured. 


Negotiate for best values based on: (1) trade-offs of 
costs and requirements licensing; and (2) 
consideration of both vendor and buyer best 
interests. 


Negotiation based on lowest cost and shortest 
schedules, endangering maturity of finished 
product. 


Overall goals: (1) obtain product at reasonable cost 
as soon as possible; and (2) achieve the business case 
for the system. 


Overall goal: Obtain lowest cost product that 
rigorously meets all requirements, but be fair. 


Relatively few review and approval steps once 
vendor is selected. 


Review and approval process more structured and 
complex once vendor selected. 


Past performance weighted heavily (sometimes 
primary factor) in selection process. 


Past performance considered, but only as a minor 
factor. 


More flexibility in vendor selection based on metrics 
and overall assessment. 


Selection of vendor forced by use of pre-defined 
metrics for proposal evaluation. 


Modifications made as procurement proceeds in 
order to get best results. 


Change difficult once process begins. 


Summary 

Very different processes with commercial much more flexible, but with no requirement for fairness, or to 
maintain the public trust. Commercial encourages vendors to offer best solution, but solution may not 
meet 100% of the requirements. Teaming and long-term relationships are more easily accommodated by 
industry. 
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Development Process 


Best Commercial Practice 


Current DoD Practice 


Vendor often tailors existing systems and uses 
COTS. System designed to fit in a defined product 
or product line architecture. 


Varies with application. Some systems use COTS. 
However, usually a new system that doesn't reuse 
legacy software. Unique systems are built with little 
regard for architecture. 


Buyer may have heavy involvement in design and 
development as part of the team (Integrated Product 
Development team). 


Formal, structured spiral, or waterfall model. Buyer 
oversees, but team approach is not usually 
emphasized. 


Reviews typically informal and stress progress 
against goals. 


Reviews usually very formal and include technical 
design details in addition to progress metrics. 


Buyer actively involved as co-participant in 
management of technical details. 


Micro management of technical details. 


Heavy user involvement. 


Limited user involvement. Heavy buyer 
involvement. 


Vendor embraces one or more industry standards 
which improves interface and integration with COTS 
products. 


Government and industry standards called out. Not 
all government standards enhanced by COTS 
products. 


Buyer requirements may be translated to more 
"general purpose" requirement for potential 
software reuse. 


Tailored system; little, if any, focus on designing in 
reusable code. 


Management reviews and degree of oversight are 
commensurate with size and risk of program. 


Notably more detailed reviews and oversight 
performed. 


Prototyping common, with joint applications 
development teams (user and developer) working to 
clarify requirements and incorporate new 
requirements that do not affect cost or schedule. 


Prototyping seldom used. 


Use of flexible architectures allows insertion and 
plug and play of COTS products. 


Use of MIL standard computers and legacy 
instruction set architectures restricts new 
development. 


Summary 

Commercial more flexible with likelihood of a team approach and is biases toward reuse and tailoring of 
existing systems. Multi-year acquisitions not re-justified each year. Product improvements are anticipated. 
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Business Practices 


Best Commercial Practice 


Current DoD Practice 


Informal contracting, joint ventures, partnerships 
with mutual economic benefit and vested interest in 
success. 


Difficult to write contract to motivate contractors to 
reduce cost; expensive to terminate contractors. 


Oversight built on established relationships. 


Burdensome cost accounting procedures required; 
extensive oversight, reporting, and documentation 
requirements. 


Can hire and fire vendors and managers. 


Government personnel regulations, policies, and 
practices determine qualifications of its managers, 
rotations of assignment, and training. 


Multi-year effort and funding. 


Multi-year effort. Yearly, unpredictable funding. 


Summary 

Commercial practice more flexible with greater incentives. 




Integration Testing and Delivery 


Best Commercial Practice 


Current DoD Practice 


Unless system is for a. new plant, then there are 
major "cut-over" issues. 


Similar "cut-over" or transition issues. 


Sometimes difficult to assemble complete system in 
laboratory environment due to cost. 


Usually integrate system in laboratory prior to 
operational testing. 

Development testing vs. operational testing via 
statutory mandate. 


Beta testing widely used to quickly find errors. 


Little beta testing. 


Ultimate acceptance authority rests with buyer, not a 
separate organization. 


Structured, specified operational testing conducted 
by separate authority. Acceptance authority rests 
with buyer. 


Buyer/user/vendor are a team. 


Emphasis on DoD as oversight and approval 
authority. 


Summary 

Integration and functional testing seem appropriate to the need. DoD use of separate test agency adds time 
and complexity. Absence of beta testing increases cost to DoD. 
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Rights in Data 


Best Commercial Practice 


Current DoD Practice 


If custom development, buyer gets all rights, but 
vendor may retain right to subsequent sales. 


Specified by contract. 

Government usually demands all rights for 
government use due to organic support and 
maintenance needs, and competition (via statutory 
mandate). 


If tailored version of standard system, buyer only 
gets rights to tailored parts. 


Same as commercial. May have exceptions for 
proprietary material. 


Summary 

Similar, but commercial is a little more flexible, especially regarding resales. 



Maintenance 


Best Commercial Practice 


Current DoD Practice 


Organic support shifting to outsourcing or vendor. 


Organic support, with reluctance to be dependent on 
vendor. Use of depot maintenance makes 
interoperability issues more manageable. Also, 
must be responsive to user for critical systems. 


Level of discrepancy reporting required is based on 
user needs. Problem resolution usually delayed 
until next major release of software. Buyer has 
limited power to implement fast resolution of 
problems. 


Bureaucratic and paper-intensive discrepancy 
reporting and change control board often imposed. 
Award fees may be tied to problem workoff, 
resulting in fast resolution of problems. 


COTS solutions take advantage of economics of 
scale since maintenance costs are leveraged across 
multiple buyers. 


Unique software requires custom maintenance all 
borne by the single buyer. 


Summary 

The DoD and industry currently have different requirements, and trends are moving apart. However, DoD 
is currently reevaluating its practices for hardware systems and perhaps should also reevaluate for software. 
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APPENDIX D 



PROPOSED ACTION PLAN FROM 
SERVICE SOFTWARE EXECUTIVE 

OFFICIALS 



JOINT RECOMMENDED PROPOSED ACTIONS 



I. DESIGNATE SINGLE, COMMON DOD SOFTWARE MANAGEMENT OFHCIAL 
(ACTION OFHCE (USD)) 

A. ESTABUSH DEPUTY ASSISTANT SECRETARY OF DEFENSE 

(SOFTWARE) 

B. PROVIDE SUFHCIENT RESOURCES FOR MISSION 

n. SOFTWARE ACQUISITION AND UFE-CYCLE MANAGEMENT (ACTION OFFICE 

NEW DASD) 

A. PROMOTE REUSE BASED ON DOMAIN MANAGEMENT 

1 . Define domains of interest or "areas of business" 

2. Assign responsibilities to manage the domains 

3. Establish and manage common architectures within the domains 

B. REDUCE MONITORING OF MATURE COMPANIES 

1 . Identify criteria for assessing/determine process maturity 

2. Consider maturity in source selection 

3. Allow sole source extension for high quality vendors 

C. PROMOTE GOVERNMENT/INDUSTRY TEAMING 

1 . Reduce Documentation Requirements to a minimum 

2. Provide for electronic deli very /evaluation/exchange 

D. MANAGE RISK 

1 . Mandate the use of market research 

2. Use metrics effectively for management 

3. Develop Risk Management Disciplines 

4. Stick with a winner/punish poor performers 

m. POUCY AND STANDARDS 

A. ADOPT COMMERCIAL STANDARDS (ACTION 0FnCE(ASD(C3I))) 

1. DoD invests in commercial standards developments 

2. Change FAR/DFAR and SD-1 as appropriate 

B. REVISE THE MILESTONES FOR SOFTWARE-INTENSIVE 
DEVELOPMENTS (ACTION OFHCE USD(A&T)/ASD(C3I)) 

1 . Address the need for a software-first philosophy 

2. Provide for a layered software/hardware standards based architecture 



DEVELOP DOD "BETA TEST" PHILOSOPHY (ACTION OFHCE OSD(T&E) 

1 . Team with Universities/other appropriate activities 

2. Allow fielding of software direct from test beds with user consent 



IV. PERSONNEL 

A. DEVELOP SOFTWARE ACQUISITION MANAGERS (ACTION OFHCE 

USD(A&T)) 

1 . Provide a career path for software managers 

2. Integrate software personnel into senior acquisition staff 

B. PROVIDE SOFTWARE EDUCATION FOR SENIOR MANAGERS 
(ACTION OFHCE USD(Ai&T)) 

1 . Develop DoD Senior Software Managers Course 
.2. Develop and provide interactive training tools for 
senior managers to perfect software management skills 

C. PUBUSH AND PROMOTE "BEST PRACTICES" HANDBOOK (ACTION 

OFHCE OSD(T&E)) 

D. ENSURE DOMAIN EXPERTISE (ACTION OFHCE MIUTARY 
DEPARTMENTS/ AGENCIES) 

V. SOFTWARE TECHNOLOGY BASE AND TRANSITION (ACTION OFHCE 
DDR«&E) 

A. PROVIDE FOR SOFTWARE TECHNOLOGY INSERTION INTO SYSTEM 
ACQUISITION 

B. INVEST IN THE DOD SOFTWARE TECHNOLOGY STRATEGY 

C. PROVIDE FOR THE EVOLUTION OF THE DOD SOFTWARE 
TECHNOLOGY STRATEGY TO CAPTURE EMERGING COMMERCIAL 
PRACTICES 



DEPARTMENT OF THE ARMY 

OFFICE OF THE SECRETARY OF THE ARMY 
^ 107 ARMY PENTAGON 
WASHINGTON DC 20310-0107 



Olftce, Director of Information 
Systems for Command, Control, 
Communications, & Computers 

MEMORANDUM FOR THE EXECUTIVE SECRETARY, DEFENSE SCIENCE BOARD 
TASK FORCE ON DOD ACQUISITION OF COMMERCIAL SOFTWARE 

SUBJECT: Proposed Action Plan with Regard to Common Changes Proposed by DoD 
Speakers to Revise the Software Acquisition Process 




The DSB Task Force asked us to provide a list of common recommended 
changes needed to revise DoD's software acquisition process. Our recommended list 
of proposed changes is provided in the enclosure to this memorandum. As we worked 
together to develop this list, we found it useful to categorize the recommended changes 
into five major categories. Several of our recommendations did not receive adequate 
emphasis in our presentations, yet are important to the process we are studying. We 
feel that if OSD is serious about addressing Software Acquisition issues, a single 
senior office with primary responsibility for effecting the reforms proposed by the DSB 
will be essential. Further, such an official or office must receive "from the heart" 
sustainment and backing from the highest levels within OSD. The enclosed list 
addresses our concerns with regards to the questions before the DSB. We hope that 
this will be beneficial to the DSB Task Force and we all look forward to continued active 
participation in this effor 




PETER A. KIND 

Lieutenant General, GS 
Director of Information 
Systems for Command, Control, 
Communications and Computers 




±CYD K. MOSEM^N II 
Deputy Assistant Secretary 
(Communications, Computer, and Support 
Systems) 



Date 



4 Feb 94 



Date 





OHFTG. HEKMAN 
Rear Admiral, SC, USN 
Commander, Naval Information 
Systems Center 



Date 



4 Feb 94 



Printad en nA Recycled Peper 



0 5 OCT 1994 
Ref: 94-F-2088- 



Mr. Jerome S. Gabig, Jr. 

Venable, Baetjer^ Howard & Civiletti 

Suite 1000 

1201 New York Avenue, NW 
Washington, DC 20005-3917 

Dear Mr. Gabig: 

This responds to your September 22, 1994, Freedom of 
Information Act (FOIA) request pertaining to the Report of the 
Defense Science Board Task Force on Acquiring Defense Software 
Commercially. Our September 30 interim response refers. 

The Defense Science Board has provided the enclosed record 
as responsive to your request. There are no chargeable costs for 
processing this request, in this instance. 



Enclosure: 

June 1994 DSB Report 

Prepared by Kahn: 4F2088L1 : 10/4/94 :DFOI :X71160 :grj^jpk ^yl ^wh 



Sincerely, 




A. H. Passarella 
Acting Director 
Freedom of Information 



and Security Review 
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Executive Summary 



The Defense Science Board Task Force on Acquiring Defense Software Commercially recognizes 
that DoD systems are becoming increasingly dependent on the use of software as the mechanism for 
implementing operational capabilities. To adapt to changing military and national security situations, DoD 
is more dependent than ever on its ability to modify mission software rapidly, often in near real-time. 
However, software remains the schedule and cost driver for the development and maintenance of many 
important defense systems. 

In its review of current DoD and conmiercial software acquisition practices, the Task Force found 
notable differences, as evidenced in Appendix C. There are, however, many similarities between the 
various categories of DoD and commercial software systems. Although there are indications that 
commercial development efforts have achieved better predictabihty and lower costs, the Task Force noted a 
significant lack of credible, quantitative data to substantiate this assessment. 

In general, the Task Force concluded that DoD's investment in software requires greater DoD-wide 
management control and oversight in the coming years if the Department is to exploit the use of 
commercial software acquisition practices fully, as well as rapid advances in software technology. The 
following is a summary of selected findings and recommendations toward that end. 

Process Credibility: Current DoD practice is not compatible with commercial business 
practices. DoD should work to make necessary changes to acquisition regulations such as: 

• Having program managers manage 3 of 3 (price/schedule/fiinctionality ) but only constrain 2 of 3 

• Defining successful performance on contracts as delivering a solution (with predictable price, 
schedule and functionality) not adherence to government processes, procedures and specifications 

• Not requiring c-level specifications for software projects developed in Ada 

• Establishing mechanisms to allow both current ability to perform as well as past performance as 
key factors in source selection 

• Encouraging offerors to demonstrate as much functionality as possible as part of bid without 
eliminating domain knowledgeable competition 

DoD Program Management: DoD program management approaches discourage the use of 
conmiercial practices. Program managers lack incentives to tailor procedures to fit individual program 
needs or to develop "corporate" solutions (e.g., employ common architecture or common software 
components). DoD should establish and implement overarching software life cycle guidelines more 
conducive to the use of commercial practices and products, such as: 

• Defining software architectures to enable rapid changes and reuse 

• Facilitating early system engineering and iterative development 

• Participating in development of commercial and intemational standards 

• Allowing the fielding of software directly from test beds with user consent 

• Requiring program managers to stay with programs at least through beta testing to maintain 
continuity of understanding of original nuances in requirements 



